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Abstract — The aim of this article is to present a method to evaluate the 
adequacy of a virtual opponent in a digital game. The motivation for this 
research arose from the lack of methodologies to evaluate a game in terms 
of its artificial intelligence (IA) applied to a virtual opponent when 
confronted with its players. The methodology for this research consisted of 
the case study applied with 17 children - 8 with intellectual disability and 
2 with hearing impairment - in a basic school in play activities with 
Dinobase digital game. Dinobase is a digital game developed to help 
children to learn about mathematics, and it was created in a universal 
design approach, by considering children with disabilities. The virtual 
opponent must identify the level of ability from children and adapted its 
behavior. On previous research, the virtual opponent was evaluated with 
personas, on this research, we use data and analysis triangulation that 
involved log records, observations, interviews, and video. The results 
show that the methodology adopted allowed to identify the needs of 
improvement to the virtual opponent, that were not shown just using 
personas. 





I. INTRODUCTION 


Apply this principle to educational digital games require to 


Digital games can help students to best understand some 
phenomena and get them engaged in classes. According to 
[1], games improve cognitive development in students 
because they propose challenges, have rules, and demand 
higher order thoughts to achieve success. In Special 
Education, particularly, digital games can provide 
experiences and allow adaptations for students according 
to their needs [2]. Activities with digital games can be 
repeated and present different degrees of difficulty, which 
favors learning. Students learn in a playful way through 
digital games since they can play it. 


The principle of Universal design, in this context, can 
be observed in order to provide a basis to the development 
of accessible games. According to Connell [3], universal 
design refers to “the design of products and environments 
to be usable by all people, to the greatest extent possible, 
without the need for adaptation or specialized design.” 
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plan the game to be accessible by children with different 
disabilities, like blindness, intellectual, deafness or 
mobility. This is not a trivial task. 


Artificial intelligence is one of the strategies to 
development of accessible digital games by adapting the 
game to the student’s cognitive abilities. This was the 
alternative adopted by Nagasava [4] in the development of 
a virtual opponent to the Dinobase digital game. This 
strategy is promising because the game adapts to the 
gamer, allow him to play in a similar level of capacity 
against the "computer", reduce frustration and enhance 
engagement. The virtual opponent must learn player 
behaviors. 


It is necessary to evaluate the effectiveness of a virtual 
opponent in adapting to the player's behavior, to achieve 
the objectives of accessibility and consequent learning. 
The use of “Personas” by Nagasava [4] shows some 
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positive results with the virtual opponent, but not uses real 
people. Strategies to evaluate games, like observations, 
interviews and logs files are proposed by [5] [6] [7], but 
they do not analyze artificial intelligence in games. 
Although some studies evaluate games, we found a lack of 
methods to conduct the analyses of the behavior of virtual 
opponents in games. This raised the need for research. 


This article has the purpose to present the method 
adopted to analyses the adequacy of a virtual opponent in a 
game. We adopt a case study of the evaluation of Dinobase 
game virtual opponent. We present in the next sessions the 
Dinobase game; the method to analyze its virtual 
opponent; the results of its analyses and the final 
considerations of this study. 


Il. DINOBASE GAME 


Dinobase is a digital game based on Base 3 analog game. 
Base 3 proposes to develop mathematical concepts of 
exponentiation, specifically the base 3. It consists of a dice 
and colored cards representing values of this base: 3°, 3!, 
3? e 3º, respectively represented by red circles, blue 
squares, green triangles, and white rectangle. On each turn 
the player throws the dice and acquire cards with values 
corresponding to the number obtained. It is winner of the 
match the player who first acquire the white rectangle (3º), 
however, as the data only has values between | and 6, to 
win the card worth 27 it is necessary exchanges cards 
while the player progresses in the game. The child needs to 
understand, for example, that to get a green triangle (37), 
he/she needs three blue squares (3!). Thus, it is necessary 
to understand the relationship among quantity and image, 
which are not clearly exposed, requiring complex process 
of abstraction. 


In digital game Dinobase the cards were substituted by 
dinosaurs’ graphics elements [8]. In Fig. 1 we see the main 
interface of Dinobase, which has the inventory of a player, 
presented in his/her turn in the game. As Base 3, in 
Dinobase there is no relationship between the figure and 
the value that it represents, neither relationship between 
the Base 3 cards with the imageries of the Dinobase. Thus, 
the required level of abstraction in both games is very 
high, making it even more complex in the case of digital 
game because the player does not have at hands tangible 
objects to manipulate. Adaptation between these games is 
not something direct because of the graphical difference 
among them, requiring an understanding of the game to 
realize that they are equal. Table 1 shows the relationship 
between values of the items and their representations in 
both games. 
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Fig. 1: Dinobase main interface 


Table 1: Values of items and their representations in Base 
3 and Dinobase games 
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Source: /8] (adapted by author) 


Dinobase 2.0 was a new version of Dinobase that allow 
children play against a virtual opponent. The virtual 
opponent was developed using the agent intelligent 
approach by implementing a Finite-state machine. The 
agent observes the player’s actions and tries to play like 
him/her, and this way children will have an opponent with 
him/her similar skills. The idea is that the game will not so 
difficult to cause frustration neither easy, to cause lack of 
interest [4]. This strategy aims to offer a game that 
intelligently adapts to the different cognitive abilities of 
children, thus leading to the concept of universal design. 


Dinobase 2.0 was available with personas. The 
technique of persona is a technique used to represent the 
profile of the group of users that will use the system, 
making possible to make implications to meet the needs of 
the target audience [9]. Nagasava [4] defined personas 
with three levels of abilities: lesser ability, intermediate 
ability and highly ability. In this way, the objective was to 
evaluate whether the virtual opponent would converge to a 
single skill level or would alternate between them, 
adapting to the skill level of the player. Tests with 
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personas showed weaknesses with the virtual opponent, as 
in the case of children who played with highly ability and 
had difficulty to win, which can cause frustration. This 
motivated the research to evaluate the game with real 
players with different skill levels, including children with 
disabilities. 


Ii. METHOD 


The method to analyses the virtual opponent of the 
Dinobase digital game had a qualitative approach. The 
research was conducted through game play workshops 
with children, including those with disabilities. We chose 
to do the tests in a controlled environment, because this 
approach allows the evaluator to have control over the 
actions of the users and to verify the environmental and 
social influences that can affect the performance of the 
actions. 


We adopt different ways of acquiring and analyze data 
with the goal to verify if the virtual opponent had its 
behavior adapted to the human gamer. According to 
Roger, Sharp & Preece [10], to collect data on user 
performance, a series of methods are used to collect data, 
as: users’ video to capture facial expressions and body 
language; system logs for capturing data through the 
system, such as mouse movement and mouse clicks; 
microphones in the environment for voice recording; 
satisfaction questionnaires or questions that ask for 
information about system features; and structured 
interviews to collect information about what they liked 
about the product and how it was used. So, we observe 
children playing the game, collect game logs, made 
interviews with children, analyzed pre-existing studies 
with Dinobase (Fig. 2). The use of diverse data in research, 
which is known as methodological triangulation, allow to 
reduce biases or deficiencies caused when we use just one 
method [11]. In the sequence, we explain each approach of 
collect data. 
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Observation Interview 


Triangulation 


Video 


Log record analyses 





Fig. 2: The data collecting in research. 


3.1 Log Record 


Data logs in software have the aim to register the user's 
relevant events while he/she operates the software. These 
data can be used to audit the software, analyze its use or 
diagnose problems. Kakeshita and Ohta [12], for instance, 
use data log to analyze students” performance in answering 
questions, their achievement level, and their learning 
process. According to the authors, the instructor uses the 
analysis to improve the educational contents. However, in 
our research, the purpose with data log register is to 
analyze if virtual opponent behavior is accordingly with 
student behavior. 


In the case of Dinobase 2.0 there was no data 
collecting, so we implemented a log register with the aim 
to store all the user's and virtual opponent's actions. This 
new version of the game is called Dinobase 2.1. Each log 
register is composed by the following structure: player 
name (VIRTUAL if it is the virtual opponent), round, date, 
action, action value, number of elements eggs, amount of 
baby elements, amount of young dinosaur elements, 
amount of adult dinosaur elements, total amount of score. 
Each log record in the log file represents one action, that 
can be: 

1. S - play the die; 

C - buy items; 

T - exchange items; 
F - end of an innings; 
V - victory; 


ON E Oo 


W - attempt to finish an innings without buy all 
possible items; 

. X - attempt to acquire items without having points; 
Y - attempt to make exchanges without selecting 
elements or not having necessary punctuation. 


on 
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In our experiment with children, Dinobase 2.1 
registered all actions performed by students and virtual 
opponent, which were analyzed and are presented at 
section Results. 


3.2 Observation 


Observation is a technic of data collection that allows 
researchers to see, to hear and to exam facts or phenomena 
that they want to study [13]. This technique requires direct 
contact with reality and allow researchers to identify and 
get proves about the person behaviors while in contact 
with the object of study. Observation as a scientific 
technique needs a plan, methodical registration, and 
verifications and controls for its validation. In our 
research, observation was conducted through a systematic 
approach, by observing students in workshops of game 
play. 

Observation was conducted by two researchers, who 
take notes on a form previously prepared. This team 
observation, according to Marconi & Lakatos [13] is 
advisable since they can observe the phenomena through 
diverse angles. So, each question from the form was 
discussed by the researchers, that way there were a 
consensus about children behavior. The questions are: 


1. Did the child present any difficulties in 
understanding the functioning of the game? 
2. Is the child unsatisfied because the game is too 


difficult? 

3. Is the child unsatisfied because the game is too 
easy? 

4. Does the child show disinterest in playing the 
game? 


5. Has the child lost focus during game play? 
6. Has the child failed to purchase items drawn before 
the end of the innings? 
7. Has the child failed to exchange available inventory 
items when possible? 
3.3 Interview 


The third approach to data collect was based on 
interviews with students after play Dinobase game. The 
interview is a meeting between two people to obtain 
information about some subject in an oral form [13]. On 
our research, interviews had the intention to get from the 
children their feelings, satisfaction, and fun playing 
Dinobase. 


The interviews were standardized and structured [13], 
so the answers could be compared. For each question, we 
write down two (2) for the answer "always", one (1) for 
"sometimes", and zero (0) if the children answered 
"never". The questions were: 


1. Did you like the game? 
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2. Would you like to play again in another 
opportunity? 
3. Were you able to defeat your opponent? 
4. Did you play against another child who was on 
another computer or played against the computer? 
This last question, in particular, aims to investigate 
how children perceive the virtual opponent. A virtual 
opponent must be as real as possible, that is, it is not 
enough to fulfill its basic function in the game, which is to 
interact with other players within the logic proposed for 
the game, it must also present natural behaviors of a 
human player, such as know the environment in which it is 
inserted, display emotions and personality as if it had a life 
of its own, with the goal of human players questioning 
themselves against who they are playing [14]. 


3.4 Video recording and analyses 


Dinobase and Base 3 were used in former research, as 
tools to help children to understand digital games engines. 
On this research, children were filmed and analyzed 
thought narrative analyses [15]. We use those materials to 
analyze children behavior in playing the game, with the 
objective of validating the profiles classified by Nagasava 
[4] and to prepare to the workshops. The identification of a 
new profile would be of great impact, having an enormous 
gain for the improvement of the virtual opponent and to 
make it more complete. 


The videos and narratives were analyzed to observe if 
during the rounds: (i) the player stopped making purchases 
even with available points and (ii) the player stopped 
making exchanges even with elements to do so. These 
behaviors show the student understanding about the game. 
The players behavior did not change during the match, due 
to the matches being short. No player presented significant 
evolution during one single match, that means, no one 
initiate a match presenting great difficulty and finished it 
with great dominion in understanding the thematic of the 
game. However, it was verified that the players acquire 
knowledge when playing several matches, that can 
influence the results of this research. 


The analysis of videos and Alves' narrative [15] 
confirmed the profiles classified by Nagasava [4] that it, 
three levels of players’ ability. This helped for the 
preparation for the workshops with children since we had 
data to assist in conducting the evaluations. 


3.5 Research place and participants 


The research was conducted in a public school in the 
city of Itajaí — SC, Brazil. The school selected was the 
Basic School José Fernandes Potter, located in the 
Espinheiros neighborhood. The choice of this educational 
establishment was due to its characteristics of specialized 
educational services, and for attending countless children 
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with disabilities in the age group defined for the research, 
that means, 8 to 10 years old. the research was authorized 
by the Education Secretary of Itajaí, Santa Catarina, Brazil 
and all ethical aspects were respected. 


The teacher of the SRM (Multifunctional Resource 
Room) - where the specialized education is conducted in 
opposite shift from regular classes - helped to define the 
group of children for the research. Her participation was 
important because Dinobase was created in the context of 
special education, notably for children with intellectual 
disabilities. To conduct the research, we needed children 
with diverse levels of intellectual ability, so we define to 
include children with and without disabilities. A total of 17 
children were selected, among them 8 with intellectual 
disability and 2 with hearing impairment. All of them 
within the age range defined for the research. Parents were 
informed about the study and agreed to the participation of 
their children through consent form. 


The workshops were held in a single day. Children 
divided into groups and invited to play Dinobase 2.1, each 
one on a computer, individually. The children received 
general instructions about the operation of the game before 
play. All actions that they performed within the game were 
saved in log files for further analysis, they were 
photographed and filmed, and we took notes on the 


observation form. The individual interview was made 
immediately after the child play the game. Fig. 3 shows 





Fig. 3: Children playing Dinobase 2.1 


IV. RESULTS 


With the objective of evaluate the virtual opponent of 
Dinobase 2.1, we performed the workshops with children, 
with and without disabilities. Too many data were 
collected on this activity, since we decided to construct a 
data triangulation so that we could conduct the results to 
have a realistic analysis about the artificial intelligence 
applied to the game. Thereby, the data analysis was 
divided into: (i) analysis of the data generated from the 
observation of children in game play activity and 
interviews with children after game play activity; (ii) 
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analysis of the videos recording with the children in game 
play activity; and (iii) analysis of the log registered during 
children’s activities. 


4.1 Analysis of children in play activity: observations and 
interviews 


Children behavior in game activity were analyzed 
through the notes made during the observation and the 
interviews with children after the game activity, with the 
intent to verify if there was any dissatisfaction generated 
by the virtual opponent that could make the child lose 
interest in the game and if they realized who was their 
opponent. 


The analyze shows that five children (29.41%) at some 
point in the matches were dissatisfied because the virtual 
opponent made it easier to play; three children (17.65%) 
demonstrated at some point their dissatisfaction with the 
game because it was very difficult, among them two are 
children with more accentuated intellectual impairment; 
the other children (52,94%) showed satisfied with the 
game. 


Looking to the difficulty to play Dinobase 2.1, five of 
the 17 children (29.42%) had difficulty in understanding 
the gameplay, eight (47.06%) presented some difficulty at 
some point; and four of them (23.52%), did not present 
any difficulties in any moment. 


About the interest in the game, just four children 
presented at some point disinterest during the matches, and 
these were the same children who presented more 
difficulty in understanding the game. Other seven children 
lost focus on the match in some moments, but the videos 
of the workshops help to observe that those distraction due 
because some external factors draw attention, not due to 
the dissatisfaction for the game. 


All children failed to make purchases and exchanges at 
some time, this shows that no child had full control over 
the operation of the game. 


The interviews with children show that they liked the 
game and would play it again, this shows that the game 
was interesting and fun for them. All children answered 
that they managed to beat the virtual opponent, but 
observation shows that some children had not been able to 
beat it. When asked who the opponent was, all children 
responded that they were playing against the "computer" 
and not against another child. When asked why, some 
children responded that they did not know the answer, 
other, however, responded that the actions the computer 
took were very fast, so it did not appear to be actions done 
by a child. 


The experiment shows that sometimes the virtual 
opponent facilitated the play, and sometimes it turns it 
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more difficult, also, children realized that they are playing 
against a software, and not another human being. Those 
results reveal that the virtual opponent behavior is not 
adequately balanced according to children’s abilities. 
According to Monteiro & Santos [14], games become 
more interesting when the player believes he is interacting 
with another human, so his actions are not so predictable, 
which makes the game more challenging and fun. To 
Ravyse, Blignaut, Leendertz et al [16] some success 
factors for a serious game include realism, interaction, and 
adaptivity. If the game cause frustration, the player may 
decide not to play it anymore, and in the case of an 
educational game, the learning intention will not be 
fulfilled. 


4.2 Video recording analysis 


Children behavior and engagement while playing 
Dinobase were analyzed through the videos recorded 
during the workshops. We adapted the Involvement Scale 
proposed by Cathcart [17], so we analyzed children's level 
of concentration, energy, complexity, facial expression, 
persistence, time to play, and their comments. The 
Involvement scale allow understanding if the child is 
engaged in the activity, which is a necessary condition for 
her/his learning. For each child, the Likert scale values 
from | to 5 (where | is very low and 5 is very high) were 
defined for each item. The results are: 


e Concentration: defines the concentration level of 
children during the game play. From 17 children, 11 
had very high concentration, 6 children had medium 
and high concentration. Those data demonstrate that 
children were engaged to play and trying to win the 
game; 


e Energy: defines child's willingness to participate in the 
workshops. Children who were quiet, shy and/or 
unwilling to attend workshops were considered with 
low energy. Many children started the workshop very 
shy because they were dealing with the researcher who 
is a stranger to them. Slowly, they were more relaxed 
and ask questions about the game, even so, children's 
energy level was assessed from low to medium. Of the 
total children, 4 received a low score, 7 received an 
average score and 6 children received a high score; 


e Complexity: child's ease in carrying out activities that 
require high mental effort. Low scores were given to 
children who demonstrated greater difficulty in 
performing tasks that required high attention. Of the 
total, 9 children received medium, low and very low 
scores, 3 of them have intellectual disabilities. Eight 
children received high and very high scores for this 
characteristic, 5 of them have intellectual disability and 
2 of them have hearing impairment; 
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e Facial expression and posture: define whether the child 
demonstrates satisfaction, doubt or astonishment in 
carrying out the tasks and whether the child maintains 
the posture during the difficulties presented in the 
game. Children received high and very high scores, 
with only 3 children receiving low scores for this 
characteristic, coincidentally they were children with 
intellectual disabilities who at some point did not 
accept that the virtual opponent was winning the match 
with some ease. The data show that the children 
maintained their posture even in times of difficulty and 
facial expression showed satisfaction in most of the 
time; 

e Persistence: defines whether the child maintains 
persistence in reaching the proposed objectives even in 
the most difficult moments. All children participating 
in the workshop received a high and very high score 
for this characteristic (14 children received a maximum 
score). This shows that children were very excited to 
participate in the workshop and overcome the 
difficulties that the game presented; 


e Accuracy: defines child's accuracy in the game 
activities, received a low score if he / she took the 
games in trial and error. Accuracy was the feature that 
received the lowest score among all the other features. 
Only 3 children received a high score, 8 children 
received a low and very low score; 


e Reaction time: defines the reaction time of children, if 
it takes a long time to perform a certain action. Even 
children with intellectual and hearing disabilities had 
good scores for this characteristic, in total, 8 children 
received high or very high scores (6 of them with 
intellectual or hearing disabilities). Only 4 children 
received a low or very low score; 


e Verbal comments: verification of the child's 
participation and interest in the workshop, whether 
with comments, interest in questioning colleagues, 
concern about making good starts. Only 5 children 
received a high or very high score for this 
characteristic. This defines that children were very 
quiet during the workshop, few of them asked 
questions about the actions or functioning of the game. 
Many children felt ashamed to ask or answer the 
questions the researcher asked. 


4.3 Log analysis 


In taxonomy - player modeling proposed by [18], Off- 
Line Review is the evaluation of a game log after its 
completion. The Off-Line Review proposes to analyze the 
different states of the player or agents applied to the game 
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to reach a set of information for the purpose of create 
strategies or improve the characters involved. 


The analysis of the logs allowed verify whether the 
virtual opponent was balanced in a way to behave 
similarly to the player, adapting to his skill level. The logs 
also sought to assess whether the virtual opponent 
facilitated or hindered the game for the player, an 
important factor for the child to have the opportunity to 
learn from the game. 


Logs of 3,147 actions were collected over 31 matches. 
Some children played more than one match because they 
had more time available for the workshop. All the clicks 
the player made during the game were considered actions, 
that is, clicks on buttons, inventory items, purchasing 
actions, item exchanges and end of the round. Clicks on 
non-clickable areas of the game were not considered. 


Virtual opponent won 9 matches out of the 31 disputed, 
that is, the percentage of victory was 29.03%. It was 
expected about 50% of victories if the virtual opponent 
had similar actions to the player, so this percentage was 
considered low. In a detailed analysis of the logs of these 9 
victories, in 2 of them the player did not win because 
he/she did not make the correct move, it means, he/she did 
not realize he/she has the points to win. The other 7 
matches the virtual opponent was very advantageous in the 
raffle of values and it would be very difficult for the player 
to be the winner. In other words: in 7 of the 31 matches 
(22.59%) the virtual opponent won easily due to having 
more luck in the raffle of values. On the other hand, from 
the 22 matches in which the player was the winner, 4 of 
them the virtual opponent already had the necessary score 
to win but it let the player win because he/she had great 
difficulty on playing. 8 of them the virtual opponent did 
not even did 21 raffle points, so the player had a lot of 
advantage in the raffle of values and the virtual opponent 
had no chance to a competitive game. These results shows 
that the luck factor of the game does not make it 
competitive. In addition, the match always starts with the 
player, causing a disadvantage to virtual opponent. 


The exchange of items to progress in the game are 
fundamental actions in Dinobase because it is through 
them that the child develops their understanding of 
mathematical concepts involved and can achieve victory in 
the game. For the 31 matches, in 20 of them (64.52%) the 
percentage of rounds that the virtual opponent failed to 
make exchanges was higher than the percentage of rounds 
that the player failed to make exchanges; in 8 of them 
(25.80%) the percentage of rounds that the player failed to 
make exchanges was higher than the percentage of rounds 
that the virtual opponent stopped making exchanges and in 
3 matches the player and the virtual opponent made the 
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exchanges whenever possible, not accumulating elements 
in the inventory. The data show that the virtual opponent 
did not act in a similar way to the player's actions. Often, 
the virtual opponent "facilitated" the game by not making 
the possible exchanges, even though the player was 
playing optimally. 


To verify the children's learning, the total of their 
actions were analyzed. The objective was to check if the 
players were understanding the game or just playing with 
trial-and-error actions. For this case, we considered the 
matches that presented more than 30% of the wrong 
actions as a learning margin. Only 6 of the 31 games were 
above the 30% learning margin. In other words, it can be 
considered that 19.35% of the matches were played 
through trial-and-error due to the children not 
understanding the real value of the elements to correctly 
carry out the purchase and exchange actions. 


The analysis of the actions in play of some children, 
revealed relevant information for the assessment of the 
virtual opponent. One of them, a hearing-impaired boy, 
was very interested in the game and played 7 matches of 
Dinobase. He won 6 matches (85.72%), in which the 
virtual opponent did not reach the necessary score for the 
victory, that is, he won easily due to being luckier on 
raffle. After a few matches, the boy realized that the virtual 
opponent was performing purchase actions like his. So, he 
started to buy only elements of lesser value, with the 
objective of deceiving the virtual opponent, making him 
only buy the items of lesser value. Through the logs it was 
possible to notice that the virtual opponent considered the 
purchase of items more preponderantly, not reproducing 
the exchange actions in the same way as the player. The 
child's perception of this behavior allowed him to "cheat" 
the software. 


Another player showed different behaviors. The boy 
with severe intellectual disability had difficulty in 
socializing, communicating, and understanding how the 
game works. From the logs of his matches, it is observed 
that the boy did not make the necessary exchanges to 
progress in the game and in most cases he’s actions were 
by trial and error. Checking the videos of the workshop, 
the player frequently asked his colleagues or the teacher 
why he was unable to perform a certain action. The boy 
played 3 matches and won 1 of them. The luck factor was 
again observed in the raffle of points. The virtual 
opponent, in this case, behaved appropriately, making few 
item exchanges, just like the player. 


4.4 Recommendations towards virtual opponent's 
Improvement 


The triangulation of data made it possible to examine 
the game Dinobase and its virtual opponent through 
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several dimensions. The analysis of children's behavior 
and their perceptions increased understanding of their 
relationship with the game, the fun promoted, and of the 
learning possibilities. The records of actions (logs) 
presented quantitative data that revealed weaknesses and 
potentialities of the virtual opponent and corroborated the 
actions of the children, observed empirically. This analysis 
made it possible to develop recommendations for 
improvements for future implementations of the game 
under study. 


e Slower presentation of the virtual opponent's actions on 
the interface: all the children realized that they were 
playing a game against the "computer". According to 
[14], if the child does not realize that he/she is playing 
against a virtual opponent, he/she may take the game 
more seriously, having a greater rivalry for thinking 
that he/she is playing against another child, then it is 
recommended to leave the agent's actions slower. With 
this measure, the player will be able to better visualize 
opponent's actions, so he/she could reflect on the play 
and promote learning; 


e Visual move: the game visually presents just the result 
of the virtual opponent's move, not allowing the player 
to view the choices of items made for the exchange, 
that is, their movements in the inventory. It is 
suggested that this move is like that of the player, that 
is, that is, that visually the selected items are "clicked" 
by the virtual opponent. Learning also takes place with 
the observation of the opponent's play, so this aspect is 
fundamental; 


e Agent perception: Children found irregularities in the 
agent behavior and so they were able to define 
strategies to defeat him. The virtual opponent did not 
realize that it is playing with the same child and 
resumes his/she is learning with each match, while the 
child improves his/her skills. Thus, it is recommended 
that the agent keep the player's information and, when 
starting again, already have knowledge of his skill; 


e Start of the match: the first round in the game always 
starts with the player, giving him an advantage over the 
virtual opponent because he has more chances to add 
up the points needed to win. To minimize this problem 
the game must make a raffle to decide who starts the 
match; 


e Virtual opponent actions: data analysis showed that 
virtual opponent does not exhibit a behavior similar to 
the player, especially when exchanging items. It is 
recommended to revise the algorithm to improve the 
perception of the way the player performs his moves 
and adapt the virtual opponent's moves to approach the 
player's skill level. 
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vV. CONCLUSION 


The objective of this paper is to investigate methods to 
analysis of the virtual opponent’s behavior in digital 
games, with the purpose of validating their suitability to 
the skill levels of players. For this purpose, the authors 
carried out a case study of the digital game Dinobase, a 
potentiation learning game proposes to develop 
mathematical concepts of exponentiation, specifically the 
base 3. 


The analysis involved the triangulation of data that was 
collected in different forms: users' video in game activities; 
system logs; voice recording; and structured interviews. 
These data were analyzed in a qualitative and quantitative 
approach, and demonstrated several weaknesses of the 
virtual opponent, for which suggestions for improvements 
were presented. 


The study revealed the need to evaluate the game under 
different aspects, emphasizing the participation of students 
with and without disabilities, and their different levels of 
skills and knowledge. The child's perception of the game is 
a major factor, however the analysis of the logs made it 
possible to observe in depth the actions, which in some 
cases contradicted the students' actions. 


The recommendations presented in this article were 
implemented in a new version of the game Dinobase [19] 
and can be 
https://univalildi.wixsite.com/univalildi/dinobase. It is 
suggested as a future work the revaluation of the game 
through the methodology proposed in this article, with the 
view to validate both the methodology and the 
effectiveness of the recommendations implemented in the 


downloaded at 


new version of the game. 
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